An electronic imaging system depends on an electronic image sensor to create an electronic representation of a visual image. Examples of such electronic image sensors include charge coupled device (CCD) image sensors and active pixel sensor (APS) devices. APS devices are often referred to as CMOS sensors because of the ability to fabricate them in a Complementary Metal Oxide Semiconductor process. Typically, these image sensors include a number of light sensitive pixels (that is, picture elements) arranged in a regular two-dimensional pattern or array of rows and columns, with each individual pixel providing a signal based on the light level of the portion of a scene image projected onto the pixel by a lens.
For reasons of size and the needed compactness, such image sensors usually include vastly more pixels than analog to digital converters (ADC) to digitize their signals. In order to save space, it is common practice to provide only enough storage devices to simultaneously read out the pixels of a single row. Consequently, the pixel signals cannot be measured, or read out, simultaneously but must wait their turn in a serial fashion. For example, in a CCD having a single ADC, the pixel signals are read out in a raster fashion: pixel-by-pixel within a row, then row-by-row within the array of pixels. The serial nature of image sensor readout directly controls the rate at which the entire sensor can be read to the bandwidth of the readout mechanism. If the read out mechanism of the image sensor can measure 50 million pixels per second, then it must take one-tenth of a second to read out a 5 megapixel image sensor. Reducing the time required to read the entire image sensor generally requires increasing power consumption for faster read out, or increasing size of the image sensor for additional read out channels. Neither increased power consumption nor increased size is desirable.
Because it eliminates mechanical components and reduces cost and space requirements, it is common practice to build an image capture system having no light blocking shutter. Such systems rely instead on an electronic shutter that works by resetting each photosensor, integrating photo-electrons, and then reading out the photosensor signal. The reset step can be accomplished by transferring residual charge from a photodiode to associated floating diffusion circuitry and then discarding the residual charge. The photo-electrons then begin accumulating in the photodiode for the prescribed integration time, at which point the charge is transferred into the floating diffusion and, in CMOS devices, is converted to a voltage. The associated voltage is then stored in a memory device such as a capacitor.
If the sensor has sufficiently low dark current and sufficiently good light shielding for the floating diffusion, then the transferred photo-electrons need not be read out immediately. Under these conditions, one can transfer the charge from all pixels at once into their respective floating diffusions and then wait for a short time as the rolling read out processed the signals row by row. Of course, for such a global transfer to work, each pixel would also need to have its own light-shielded floating diffusion.
An alternative image sensor readout arrangement, provided particularly by APS image sensors, permits exposure and readout of the image sensor to occur progressively row-by-row across the rows of the image sensor. This “rolling shutter” sequence avoids the differential exposure problem that the interlaced fields of a CCD exhibit by making the exposure for each row the same length of time.
As an additional advantage, the rolling shutter sequence simplifies sensor component design, since shielded storage is not required for each pixel. However, since the exposure for each row is independent from the exposures of the other rows and occurs in a sequential (or rolling) fashion with the exposures of the other rows, each row captures its portion of a scene image at a slightly different time.
Consequently, relative motion between the scene (or elements of the scene) and the image sensor causes objects within the scene to appear distorted in the image captured by the image sensor. This effect, termed image “shear”, is characteristic of rolling shutter arrangements. For example, if such a so-called rolling shutter or electronic focal plane shutter image sensor is used to capture an image of a car moving horizontally, the car moves relative to the image sensor as each row of the captured image is exposed and read out, so each row of the captured image shows the car at a different position. This can cause the round tires of the car to appear oval, and the car's rectangular windows to appear to be parallelograms. This distortion is a direct consequence of the amount of time required to read out all the rows of the image sensor. If the rows can be read at a faster rate, then this distortion can be reduced. As noted previously, however, increasing the readout rate generally requires an increase in cost and power consumption for the image sensor.
For silicon-based image sensors, the pixels themselves are broadly sensitive to visible light, permitting unfiltered pixels to be suitable for capturing a monochrome image. For capturing color images, a two-dimensional pattern of filters is typically fabricated on the pattern of pixels, with different filter materials used to make individual pixels sensitive to only a portion of the visible light spectrum. An example of such a pattern of filters is the well-known Bayer color filter array (CFA) pattern, as described in U.S. Pat. No. 3,971,065. Though the Bayer CFA has advantages for obtaining full color images under typical conditions, however, this solution has been found to have its drawbacks. Filtering to provide narrow-band spectral response tends to reduce the amount of light reaching each pixel, thereby reducing the light sensitivity of each pixel and reducing pixel response speed.
As solutions for improving image capture under varying light conditions and for improving overall sensitivity of the imaging sensor, modifications to the familiar Bayer pattern have been disclosed. For example, commonly assigned U.S. Patent Applications Publication No. 2007/0046807 entitled “Capturing Images Under Varying Lighting Conditions” by Hamilton et al. and Publication No. 2007/0024931 entitled “Image Sensor with Improved Light Sensitivity” by Compton et al. both describe alternative sensor arrangements that combine color filters with panchromatic filter elements, interleaved in some manner. With this type of solution, some portion of the image sensor detects color; the other panchromatic portion is optimized to detect light spanning the visible band for improved dynamic range and sensitivity. These solutions thus provide a pattern of pixels, some pixels with color filters (providing a narrow-band spectral response) and some without (unfiltered pixels or pixels filtered to provide a broad-band spectral response).
Using a combination of both narrow- and wide-spectral band pixel responses, image sensors can be used at lower light levels or provide shorter exposure times. See Sato et al in U.S. Pat. No. 4,390,895, Yamagami et al in U.S. Pat. No. 5,323,233, and Gindele et al in U.S. Pat. No. 6,476,865.
Even though image sensors that employ narrow-band and broadband color filters can provide improved light sensitivity or photographic speed, some problems and limitations persist. Interline CCDs used in digital still cameras generally employ a mechanical light blocking shutter during readout to avoid charge blooming in bright areas of the scene or to accommodate an interlaced vertical CCD. Consequently, the shutter open and close times must be considered when capturing a sequence of images, necessarily limiting exposure time and sequence image capture rate. As for CMOS APS devices, rolling shutter artifacts appear even where reading speed is increased over conventional timing methods.
Thus, it can be seen that there is a need for improved readout methods that yield faster pixel response times and thus reduce motion-related aberrations, without compromising overall color sensing performance.